Many businesses rely on some form of WORM (write once, read many) data storage to store critical data. In many industries, such as the financial services and healthcare industries, businesses are required by strict records-retention regulations to archive important data, such as emails, transaction information, patient records, audit information, as well as other types of documents and data. Such records-retention regulations include, for example, Securities Exchange Commission (SEC) Rule 17a-4 (17 C.F.R. §240.17a-4(f)), which regulates broker-dealers; Health Insurance Portability and Accountability Act (HIPAA), which regulates companies in the healthcare industry; Sarbanes-Oxley (SOX), which regulates publicly traded companies; 21 C.F.R. Part 11, which regulates certain companies in the life sciences industry; and, DOD 5015.2-STD, which regulates certain government organizations; etc. Regulatory agencies generally recognize the ability of WORM data storage to ensure the permanence of archived data and therefore often require that only non-erasable, non-rewritable WORM storage be used for meeting their regulations.
In addition, many businesses place a premium on protecting certain business records or critical data files from accidental or intentional alteration or deletion. WORM functionality such as non-erasable and non-rewritable data storage can ensure long-term data permanence, even where regulatory compliance does not require the use of WORM storage.
Most current WORM implementations are based on older optical disk technology with limited storage capacity, slow data throughput, and substantial management overhead. Current WORM storage options include optical WORM platters, which each hold about 30 GB of data, and WORM tape, with each cartridge able to store about 50 GB of data. Traditional WORM storage vendors have circumvented the capacity limitations of individual media by implementing expensive, complex media library and jukebox solutions that house numerous media. Although this solution addresses capacity limitations, it creates the problem of ever-increasing management overhead for the volumes of full media removed from the library or jukebox.
As data storage technology advances, magnetic storage devices, such as magnetic disks, have become the preferred storage devices over optical disks for WORM implementations. One advantage of magnetic storage devices is the shorter time needed to retrieve data from the magnetic storage devices, because optical disks generally take a longer time to load before the optical disks can be searched. Magnetic storage devices also provide fast random access. Other advantages of magnetic disks include their generally higher capacity and lower cost. Moreover, it is relatively easy and economical to replicate the data stored in a magnetic storage device and to store the copy in another location. The ease in replication facilitates disaster recovery of the data storage system.
To address issues faced by growing business requirements for WORM data storage and to alleviate problems inherent with traditional WORM storage solutions, Network Appliance, Inc. of Sunnyvale, Calif., introduced a technology called SnapLock on its Filer and NearStore storage systems. SnapLock technology allows companies to implement the data permanence functionality of traditional WORM storage in an easy-to-manage, fast-access, low-cost, magnetic disk-based solution. Among other features, SnapLock allows a user to designate certain data files or volumes as WORM data, using the file system interface of the Data ONTAP® operating system. SnapLock provides the capability to ensure that when data are committed to WORM state, it cannot be altered or deleted. “WORM state”, therefore, as the term is used herein, is defined as a non-deletable, non-modifiable state.
In some instances it may be desirable or necessary to migrate WORM data from one set of storage media to another. For example, a business may wish to move stored WORM data from an older set of storage media to a newer set of storage media, for the purpose of upgrading the storage system. The migration may be accomplished using a well-known data copying mechanism, such as that provided by Network Data Management Protocol (NDMP). However, migration of WORM data creates the problem of how to verify, after the data have been migrated, that the data have been stored at the destination accurately, in their entirety, and are still in WORM state.
One known technology used for WORM storage provides a way to verify data by assigning a digital signature to each individual data record (this technology is not file system based). A proprietary protocol is used to store data or to migrate data between storage media. Only an entity which has the digital signature and which implements the proprietary protocol can verify the data. Effectively, this generally means that only the storage system which created the digital signature can verify the data. This characteristic makes it essentially impossible to migrate WORM data to a storage system or set of storage media that uses storage technology different from that used to store the data. Furthermore, the digital signature is not readable by a human being.
What is needed, therefore, is a technique for use in a storage system to allow migration of WORM data while avoiding the above-mentioned problems associated with WORM data migration.